Limit repo size#21820
Conversation
…or later see what is still needed
|
Does it count also LFS? (currently repo size is git repo size + LFS object size) Imho we should finally split to have two repo sizes in db (git repo size and LFS object size) and both should have different limits settable. |
|
@lafriks it does count the LFS now and the repo limit applies to both, which I think is correct. |
6f61259 to
75585de
Compare
|
LFS size is not counted. And LFS is tricky because we may end up with LFS files uploaded and the pointer push is denied because of the limit or (worse) the pointer push is allowed and the file upload fails afterwards because of the limit. |
|
Ok will look into this further. Thank you! |
|
Yes that's why my suggestion was for those to be different limits and splitting also how sizes are saved in database for repo |
|
Hi @techknowlogick @kdumontnu I have fixed the opencollective link to make it easy to donate. I've moved the funds I was able to gather 5$ - there :) Here is the new link for this activity: https://opencollective.com/oss-code-ge/projects/gitea-limit-repository-size |
|
Hi @DmitryFrolovTri |
|
@DmitryFrolovTri I've heard back from OC, and because you are not using them as a fiscal sponsor that's why I couldn't make the transfer. So I will just mention it here that upon completion of this PR/issue we will pay out $500 from our collective for the bounty (this amount was chosen to limit tax burden to whoever gets paid out), and @sapk for your work so far, if you are interested, we can pay you for your work so far (reach out to me via email and I can get you sorted). |
|
@techknowlogick I've since then re-registered and the link above is with the opencollective (OC host) now. |
…ts/integration/size_limit_test.go update
Carefully merged everything in this folder from upstream/main removing anything I had with exception to one file I did modify. Should be correct @silverwind |
|
@lafriks consider unblocking. |
| func traceBatchDecision(rc *requestContext, op, msg string, args ...any) { | ||
| prefix := fmt.Sprintf("LFS[BATCH][%s/%s][op=%s] ", rc.User, rc.Repo, op) | ||
| log.Trace(prefix+msg, args...) | ||
| } |
There was a problem hiding this comment.
Why a log-only function?
| if ctx.Data["Err_Repo_Size_Limit"] != nil { | ||
| ctx.RenderWithErr(ctx.Tr("admin.config.invalid_repo_size", ctx.Data["Err_Repo_Size_Limit"]), | ||
| opts.TplName, nil) | ||
| return | ||
| } | ||
|
|
||
| if ctx.Data["Err_LFS_Size_Limit"] != nil { | ||
| ctx.RenderWithErr(ctx.Tr("admin.config.invalid_lfs_size", ctx.Data["Err_LFS_Size_Limit"]), | ||
| opts.TplName, nil) | ||
| return | ||
| } | ||
|
|
||
| if ctx.Data["Err_Repo_Size_Save"] != nil { | ||
| ctx.RenderWithErr(ctx.Tr("admin.config.save_repo_size_setting_failed", ctx.Data["Err_Repo_Size_Save"]), | ||
| opts.TplName, nil) | ||
| return | ||
| } | ||
|
|
There was a problem hiding this comment.
Why in RenderRepoSearch, and these are "admin" related?
| gitSizeMax, err := setting.ParseRepositorySizeLimit(form.GitSizeMax) | ||
| if err != nil { | ||
| ctx.Data["Err_Git_Size_Max"] = form.GitSizeMax | ||
| explore.RenderRepoSearch(ctx, &explore.RepoSearchOptions{ | ||
| Private: true, | ||
| PageSize: setting.UI.Admin.RepoPagingNum, | ||
| TplName: tplRepos, | ||
| OnlyShowRelevant: false, | ||
| }) | ||
| return | ||
| } |
There was a problem hiding this comment.
Use form-fetch-action or link-action, and JSON response.
| var ( | ||
| lfsPointerMarker = []byte("version https://git-lfs.github.com/spec/v1") | ||
| lfsOIDRe = regexp.MustCompile(`(?m)^oid sha256:([0-9a-f]{64})$`) | ||
| lfsSizeRe = regexp.MustCompile(`(?m)^size ([0-9]+)$`) | ||
| ) |
There was a problem hiding this comment.
There should already be functions like ReadPointerFromBuffer
| // This is the number of workers that will simultaneously process CalculateSizeOfObject. | ||
| const numWorkers = 10 |
There was a problem hiding this comment.
I really don't think it's right to start 10 git processes in one push action.
| for _, packFile := range packFiles { | ||
| log.Trace("Processing packfile %s", packFile) | ||
| // Extract and store in cache objectsSizes the sizes of the object parsing output of the `git verify-pack` command | ||
| output, _, err := gitcmd.NewCommand("verify-pack", "-v").AddDynamicArguments(packFile).WithDir(dir).WithEnv(env).RunStdString(ctx) |
There was a problem hiding this comment.
Will it be very slow on large repo?
There was a problem hiding this comment.
Yes, it is extreamely slow, it will shutdown the whole Gitea instance.
~/work/gitea/.git/objects/pack$ time git verify-pack pack-7b1d851cf9f2f68ab601a8c44194c03332818497.pack
git verify-pack pack-7b1d851cf9f2f68ab601a8c44194c03332818497.pack 20.47s user 0.84s system 162% cpu 13.093 total
~/work/gitea/.git/objects/pack$
There was a problem hiding this comment.
Considering this is very slow, probably because of git has to unpack the the pack file in pre-receive (and doing it again after exit 0?) my naive thought would be lossly checks based on filesize(pack-7b1d851cf9f2f68ab601a8c44194c03332818497.pack). Calculating filesize is cheep, unpacking a large pack file is expensive.
A feasibility check of my random thoughts about disk size based limits would have to be done first...
wxiaoguang
left a comment
There was a problem hiding this comment.
I don't think the implementation can really work in production
@go-gitea/maintainers please help to improve.
|
@silverwind : although there are AI tools, please make sure the code and design are overall right , before asking AI to review or approve. |
|
Yeah I know, AI mostly just catches surface-level issues, not design issues. |
|
I think the abstraction level in this PR is too low, which may make it difficult to extend in the future. |
|
Well this means we are stuck here somewhat. @lunny @wxiaoguang could you steer a bit then?
but for 1) need some idea - mine is to drop this PR and do something forgejo style. Shall we go the direction of: |
|
I don't know, I don't need this feature and haven't really looked into details. I was just pointing out some essential design problems. |
|
@DmitryFrolovTri I think I could exercise Claude on this. Are you ok if I push fixes for all remaining discussions here? If you have a qualified agent available yourself, you can do it yourself too. |
For this complex task, you need to make sure you fully understand what you are doing, but not just blindly trust AI. |
Fully agree to this |
|
@silverwind lets reenable |
Good to see |




The goal of this PR is to define a repo limit size using "no-worsen" approach (allow things that do not increase or decrease the size on disk eventually) I think org and user level restriction could come later.
Thanks to @sapk as this is a continuation of his work that was started in #7833
Would address (except for LFS): #3658
Jan 22, 2026 Updated - no more DB limit. Only config and a run-time switch in UI for admin.
**Screenshots:**
global limit setting for admin resets after restart should manage in configindividual repo size limit for a repository admin (visible to a user who is owner of the repo)

app.ini with settings
how currently push is rejected if limit is reached

TODO: (1)
Jan 22, 2026 Removed
- [x] Add ability to have the feature on or off in the config (repository section in config), default - FALSE - A boolean parameter in config to enable the size-feature checking. If it is enabled then size limit is enforced. There is a field to edit size limit in the repo settings UI. The size limit is taken into account. If disabled then repo size limit is ignored. We can leave the size limit field in the repo settings UI and allow to edit it, however, the color of size limit value should be grayed. (telling that it is not working)the config parameter - ENABLE_SIZE_LIMIT = true/false
- [x] Add ability to have a global repo limit per installation that is enforced unless an individual repo size limit present (repository section in config) - DEFAULT: 0 - global default size limit for any repo where individual size limit is not defined. if 0 - no limit if >0 limit is set so that if a repo has 0/undefined sizelimit this configuration limit will be used instead. Config parameter REPO_SIZE_LIMIT = XXXX (should accept bytes, K, and M, and G, may be T like 1000 - 1000 bytes, 1K - 1 kilobyte. 1M - 1 M megabyte.)TOFIX: (2)
Deletion of file from UI trigger 500 when repo is over. -> TODO catch this specific error.
2019/08/16 05:23:58 ...uters/repo/editor.go:432:DeleteFilePost() [E] DeleteRepoFile: git push: remote: Gitea: new repo size is over limitation 10000 To /home/sapk/go/src/code.gitea.io/gitea/data/repositories/sapk/test.git ! [remote rejected] d9629b41f9c58da756cf806aabf5811b1ff45b50 -> master (pre-receive hook declined) error: impossible de pousser des références vers '/home/sapk/go/src/code.gitea.io/gitea/data/repositories/sapk/test.git'Creation of branch from UI trigger 500 when repo is over. -> TODO catch this specific error.
2019/08/16 05:28:42 ...uters/repo/branch.go:287:CreateBranch() [E] CreateNewBranch: Push: exit status 1 - remote: Gitea: new repo size is over limitation 10000 To /home/sapk/go/src/code.gitea.io/gitea/data/repositories/sapk/test.git ! [remote rejected] test -> test (pre-receive hook declined) error: impossible de pousser des références vers '/home/sapk/go/src/code.gitea.io/gitea/data/repositories/sapk/test.git'Console operations when failing doesn't provide a correct error message - now it does
Unable to push anything that decreases size of repo and a relevant test scenario - now is possible
Documentation update
Add a delete test - gitea should accept any action (tag, branch, commit, file) that should reduce the repo in size. Since delete doesn't remove any data, we hope for git gc to reduce size after delete eventually. Ideally we should also run git gc and confirm size release.
Added usage of
git verify-pack -vto speed up size assessment, added usage ofgit batch-check;git cat-file -snow uses workers. If pack files are present then compressed size is used as it is closer to real disk usage on both repo analysis and push analysis.Update algorithm to calculate on incoming push lfs object sizes and exclude their size from the push size (LFS objects are included into push size by git by default)
Jan 22, 2026 Removed
- [x] Rework Size Checking enablement to use only REPO_SIZE_LIMIT from config -1 - not enabled, ignore any size limits, 0 - no global size limit, but respect the limit of repo, any other value - global size limit is set, but respect the limit set on repo individually. (After LFSSize and GitSize split update the code of size checking to exclude the LFS for now DmitryFrolovTri/gitea-limit-repo-size#18)Remove any mention of the EnableSizeLimit parameter. (Remove any mention of the EnableSizeLimit parameter. DmitryFrolovTri/gitea-limit-repo-size#20)
Update code to use GitSize value from repo instead of Size. (Update code to use GitSize value from repo instead of Size. DmitryFrolovTri/gitea-limit-repo-size#21)
Optional: Update the repo size field in UI should be with human editable text (The Size limit field in the repository overview should use the FileSize and there should be no scroll for numbers DmitryFrolovTri/gitea-limit-repo-size#19)
Jan 22, 2026 Removed
- [x] Add ability to count LFS Size towards Repo Size for those who store LFS on the same disk as the repo itself.Note: If functionality to control size is enabled and a push triggers a size check (it's size would breach the limit) then push will be accepted only if total size of not referenced objects (removed size) is over or equal to total size of newly added objects in push. Or in other words pushes are accepted if they are under or at the limit after the operations they do OR if they are breaching the limit only if they don't grow disk space. Since git controls when not referenced objects are purged and it's not fast this condition could last for a while. Administrator of instance could speed it up via following steps:
<path_to_gitea_server_folder>/data/gitea-repositories/<user>/<repository>:This would free up all not referenced objects and update repository size in UI. On next push (if push size is smaller then limit) adding new objects will be allowed.
NEXT PR (4)
EVEN further PR (5)
Related: #3658 #7833